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FOREWORD 


This  report  documents  the  progress  made  to  date  in  the  Authoring  Instructional 
Materials  (AIM)  project.  This  work  was  funded  as  part  of  the  Advanced  Development 
Project  entitled  Authoring  Instructional  Materials  (Ingram  Element  Number  63720N,  Work 
Unit  Number  R-1772-ET003).  The  project  originated  in  response  to  an  operational 
requirement  promulgated  by  the  Chief  of  Naval  Education  and  Training  entitled  Authoring 
Instructional  Materials  (AIM)  System  (OR-Z-0992). 

The  AIM  project  has  developed  prototype  software  tools  for  Navy  curriculum 
developers  and  has  integrated  these  tools  with  off-the-shelf  hardware  and  software.  These 
prototype  tools  assist  in  the  production  of  both  graphic  and  textual  curriculum  materials. 

This  report  describes  how  these  software  tools  were  developed  and  used  at  several  Navy 
training  test  sites.  Based  on  user  feedback,  these  tools  are  being  revised  and  integrated  into 
a  more  complete,  operational  system  capable  of  not  only  curriculum  analysis,  design,  and 
development,  but  also  life-cycle  maintenance. 

This  report  is  intended  primarily  for  the  training  community  in  the  Department  of  the 
Navy. 


B.  E.  BACON 
Captain,  U.S.  Navy 
Commanding  Officer 


J.  S.  MCMICHAEL 
Technical  Director 
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SUMMARY 


Background 

Several  factors  are  leading  the  Navy  to  make  the  design,  development,  production,  and 
maintenance  of  curriculum  materials  more  efficient  and  cost-effective.  These  factors  include 
the  following: 

•  The  Navy  conducts  more  than  7200  courses  over  a  wide  geographical  area,  and  this 
leads  to  difficulty  in  sharing  resources  and  thus  to  duplication  of  effort. 

•  Experienced  personnel,  both  the  enlisted  technical  experts  and  the  civilian  educational 

specialists,  are  not  readily  available  for  the  curriculum  development  process. 

•  Instructors  are  often  not  given  the  training  or  time  to  become  proficient  in  the 
educational  design  of  curriculum  materials,  and  so  the  instructional  quality  of  Navy 
training  material  could  be  improved. 

•  The  development  standards  change  frequently,  and  surface  level  details  are 
emphasized.  Even  when  standards  are  stable  for  some  time,  they  do  not  fully 
describe  what  to  do  to  develop  effective  instruction. 

•  The  curriculum  planning,  development,  and  maintenance  process  is  a  complex  one  with 
extensive  record-keeping  requirements. 

•  The  managers  and  developers  in  Navy  training  activities  have  few  up-to-date  office 

automation  tools  to  use  to  do  their  jobs  more  easily  and  efficiently. 

Purpose 

The  overall  goal  of  Authoring  Instructional  Materials  (AIM)  is  to  provide  automated 
tools  for  the  analysis,  design,  development,  production,  and  maintenance  of  curriculum 
materials.  We  first  worked  on  increasing  the  efficiency  of  curriculum  writers  by  automating 
much  of  the  mechanics  of  document  preparation,  and  this  report  describes  our  progress.  We 
also  outline  our  plans  for  enhancing  the  quality  of  instruction  in  subsequently  available  tools 
that  will  be  part  of  AIM. 

Approach 

The  approach  of  the  AIM  project  was  one  of  rapid  prototyping,  which  involved  putting 
computer  systems  and  the  prototype  software  tools  into  the  hands  of  Navy  curriculum 
developers  as  soon  as  possible.  Two  test  sites  were  established  during  the  first  year  of  the 
project,  and  additional  sites  have  been  added  each  year  so  that  there  are  currently  10  sites, 
including  Naval  Training  Center,  Great  Lakes;  Naval  Submarine  School,  New  London;  and 
the  Trident  Training  Facilities  in  Bangor  and  Kings  Bay. 


The  AIM  prototype  software,  designed  and  developed  in-house  by  the  AIM  project 
team,  assists  Navy  curriculum  developers  in  the  preparation  of  training  documents  to  'support 


the  Navy  standards,  MIL-STD  1379C  and  DoD-HDBK-292.  The  AIM  software  modules 
help  to  produce  personnel  performance  profile  (PPP)  tables  and  training  path  system  (TPS) 
documents  for  analysis  and  design,  and  in  the  output  of  instructor  guide  and  trainee  guide 
materials.  Thus,  the  AIM  software  automates  much  of  the  mechanics  of  the  document 
preparation  necessary  for  curriculum  development. 

The  AIM  software  operates  in  a  hardware  environment  that  consists  entirely  of  off-the- 
shelf  (non-developmental)  hardware  items:  a  Unix  woricstation/server,  a  laser  printer, 
ASCII  video  terminals,  Ethernet  and  telecommunications  equipment,  graphics  and  text 
(OCR)  scanners,  XT-  or  AT-level  PCs  (such  as  the  Zenith  Z-248),  hardcopy  color  output 
devices  (8-pen  color  plotters  and  35mm  film  recorders),  and  other  associated  components. 
Within  this  hardware  environment,  we  also  made  available  several  off-the-shelf  software 
tools,  such  as  text  editors,  paint  programs,  and  computer-assisted  drafting  (CAD)  programs 
(CADKey  and  TekniCAD)  for  color  graphics  production. 

Initial  Tests  and  Status 

The  AIM  prototype  software  is  currently  in  use  at  each  of  the  AIM  test  sites  and  has 
been  used  to  date  to  produce  approximately  64  courses,  totaling  about  500  weeks  of 
instruction.  Some  representative  courses  are  Electrician’s  Mate  "A"  School,  Heat  Stress 
"F"  School,  Mess  Management  Specialist  "C"  School,  and  Propulsion  Engineering  Basics. 
AIM  test  sites  generally  report  time  savings  of  from  20  to  50  percent  to  produce  better  and 
more  consistent  instructional  materials.  We  learned  much  about  the  user  interface  and 
functional  requirements  of  this  prototype  software  during  its  iterative  development,  and 
these  lessons  learned  are  being  applied  to  current  AIM  software  development. 

Next  Steps 

The  Navy  faces  a  number  of  challenges  in  meeting  the  requirements  of  producing  and 
managing  its  classroom  training  materials.  To  meet  these  demands,  the  prototype  AIM 
software  is  undergoing  substantial  revision  into  a  production-level  system.  This  more 
complete  system  will  support  all  of  the  Navy’s  core  curriculum  development  components, 
such  as  both  analysis/design  paths  (PPP/TPS  and  manpower,  personnel,  and  training 
analysis  report,  MPTAR),  student  achievement  testing,  and  automatic  generation  of  tracking 
documents.  Moreover,  because  of  its  underlying  relational  database,  the  production-level 
AIM  will  facilitate  maintenance  of  curriculum  materials  over  their  long  life  time.  Thus,  the 
eventual  AIM  system  will  allow  Navy  curriculum  developers  to  efficiently  produce 
standardized  training  materials  that  are  technically  accurate  and  consistent,  to  maintain  this 
technical  accuracy  by  updating  the  electronic  databases  underlying  the  course  materials,  and 
to  distribute  these  text  and  graphic  training  materials  over  local  and  wide-area  networks. 
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Introduction 


Problem  and  Background 

Training  and  education  in  the  U.S.  Navy  is  a  massive  undertaking,  and  technology  is 
being  applied  to  make  Navy  training  more  efficient.  Several  characteristics  of  Navy  training 
and  education  are  important  to  the  automation  of  the  curriculum  development  process.  This 
discussion  will  concentrate  on  technical  training  in  the  Navy,  since  that  type  of  training 
accounts  for  the  vast  majority  (about  90%)  of  Navy  training.  In  a  typical  year,  over  3(W,000 
enlisted  personnel  are  involved  in  formal  training,  and  this  figure  includes  about  100,000 
recruits  going  through  recruit  training. 

Numerous  Widely  Dispersed  Courses 

One  of  the  most  striking  features  of  Navy  training  is  the  very  large  number  of  courses 
that  are  spread  out  over  virtually  the  entire  country  and  all  of  the  Navy’s  regular  ports.  As  of 
mid- 1986,  7200  classroom  courses  were  conducted  in  shore -based  education  and  training 
activities  (Taylor,  Ellis,  &  Baldwin,  1988),  and  about  200  classroom  courses  are  being  added 
every  year.  Some  courses  are  eliminated  but,  because  courses  are  often  equipment  specific 
and  ships  have  a  long,  useful  life,  courses  tend  to  remain  in  the  Navy  training  system  for  a 
long  time.  These  courses  range  in  length  from  a  few  days  to  over  12  months,  with  the  vast 
majority  lasting  from  two  to  four  weeks.  The  budget  for  operating  this  shore-based  training 
establishment  exceeds  $1  billion  each  year;  therefore,  making  the  training  and  education 
process  more  efficient  could  have  dramatic  cost  savings. 

One  of  the  consequences  of  this  highly  dispersed  training  environment  is  the  difficulty  in 
sharing  resources-both  training  resources  and  the  technical  design  data  on  which  the 
training  materials  are  based.  When  old  courses  are  revised  or  new  ones  are  developed,  many 
of  the  parts  tend  to  be  custom  designed  rather  than  reused  from  earlier,  similar  courses.  This 
resource  sharing  problem  actually  has  three  parts: 

1.  Curriculum  developers  do  not  have  access  to  existing  training  materials. 

2.  Even  with  access,  the  materials  do  not  reside  in  any  sort  of  database  to  ease  the 
task  of  locating  the  reusable  units  such  as  safety  lessons,  basic  drawings,  etc. 

3.  Curriculum  developers  have  no  means  of  modifying  training  materials  that  are  close 
to  what  they  want  without  redoing  the  whole  set  of  documents. 

Scarce  Personnel 

A  second  feature  of  Navy  training  is  the  difficulty  in  obtaining  the  experienced  personnel 
that  are  needed  to  develop  proper  training  materials.  To  develop  good  training  material,  at 
least  two  types  of  experts  are  needed:  experts  on  technical  content  (experienced  officers  and 
senior  enlisted  personnel)  and  experts  on  the  educational  design  and  development  of  course 
materials.  Both  types  of  experts  are  in  short  supply  at  Navy  training  activities.  Experienced 
officers  and  senior  enlisted  personnel  are  very  much  in  demand  to  man  the  Navy’s  fleet  of 
almost  600  ships.  And  the  Navy’s  civilian  educational  specialists  are  difficult  to  hire,  train, 
and  retain. 
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The  effects  of  this  scarcity  of  experts  are  shown  in  the  variable  technical  quality  of  the 
training  materials  and  the  insufficient  attention  sometimes  paid  to  the  educational  aspects  of 
the  curriculum  materials.  These  quality  control  problems  were  noted  in  a  GAO  report  (GAO, 
1984)  that  examined  apprentice  level  training  for  several  enlisted  ratings  that  were 
characterized  by  the  static  nature  of  the  technical  content.  That  is,  the  GAO  investigators 
avoided  content  areas  that  were  frequendy  in  a  state  of  flux  because  of  the  high-technology 
equipment  used.  They  found  that  training,  even  with  stable  content,  often  contained 
extraneous,  unnecessary  informarion  and  left  out  job-specific  tasks.  The  GAO  investigators 
recommended  that  more  attention  be  paid  to  the  front-end  analysis  and  design  of  curricula  to 
ensure  that  the  training  materials  contain  only  the  information  necessary  for  competent  job 
performance  and  thus  keep  down  the  cost  of  training  materials  preparation. 

Curriculum  Development  Expertise  Requires  Training  and  Time 

A  third  characteristic  of  Navy  training  policy  is  that,  while  some  special  training  is 
considered  necessary  for  an  individual  to  become  an  instructor  in  a  school,  little  is  available 
for  curriculum  development.  As  part  of  the  regular  ship-to-shore  rotation,  an  experienced 
technician  will  be  assigned  to  a  training  command  for  two  to  three  years  with  a  few  weeks  of 
instructor  training  school  and  perhaps  (but  not  usually)  a  week  or  two  of  instruction  in 
curriculum  development.  After  some  short  period  of  supervision  by  the  senior  instructors,  the 
technician  is  considered  ready  to  assume  the  regular  responsibilities  of  teaching,  which  may 
include  spending  a  good  deal  of  time  developing,  revising,  and  maintaining  course  materials. 

Thus,  technical  expens  often  are  not  given  the  training  or  time  to  become  more 
sensitive  to  the  educational  demands  of  the  job.  Especially  as  the  equipment  becomes  more 
complex  and  sophisticated,  the  memory  and  problem-solving  skills  required  for  good  job 
performance  become  more  and  more  difficult  to  teach.  This  sometimes  results  in  curriculum 
materials  of  variable  quality.  A  recent  systematic  examination  of  the  quality  of  instruction  in 
Navy  classrooms  showed  that,  for  instance,  over  50  percent  of  the  training  objectives  were 
inappropriate  for  future  job  requirements  (Taylor  et  al.,  1988).  Other  ways  of  improving 
classroom  materials  and  practices  were  also  noted. 

Frequently  Changing  Standards 

A  fourth  feature  of  Navy  training  is  that  the  standards  for  developing  instructional 
material  have  frequently  changed.  By  standards,  we  refer  to  the  proceduralized  guidelines 
that  specify  what  tests,  graphics,  student  materials,  instructor  guides,  etc.  should  be 
produced  and  how  they  should  be  developed.  These  standards  drive  both  in-house  and 
contractor-developed  training  materials,  seem  to  be  drastically  modified  every  3  or  4  years 
by  the  Navy,  and  have  changed  some  100  times  across  all  of  the  military  services  since  the 
end  of  World  War  II  (Montemerlo  &  Tennyson,  1976).  Thus,  curriculum  developers  often 
lack  adequate  time  to  become  expert  in  using  one  set  of  standards.  Even  when  the 
standards  remain  static  for  a  few  years,  they  only  indicate  what  to  do,  not  how  to  do  it- 
much  less  how  to  do  it  well  (Montague,  Wulfeck,  &  Ellis,  1983). 
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Complexity  of  Instructional  Planning  and  Development 

Another  characteristic  of  Navy  training  is  the  complexity  of  planning  and  developing 
curriculum  materials  in  a  systematic  manner.  Currently,  the  Navy  uses  MIL-STD  1379C  for 
specifying  contract  deliverables  and  the  Handbook  for  Training  Materials  Development 
(DoD-HNDK-292)  as  guidance  for  Navy  personnel.  The  handbook  consists  of  over  500 
pages  in  2  volumes  and  contains  over  50  complex  multipage  flowcharts,  some  of  which  are 
more  than  10  feet  in  length.  The  handbook  provides  direction  on  how  to  develop  curricular 
materials  and  the  extensive  supporting  documentation  that  must  be  generated  to  support  the 
development  of  even  a  small  (two  to  three  day)  course.  One  of  its  effects  is  to  require  a  large 
amount  of  record  keeping  for  the  people  developing  and  managing  instructional  documents. 

This  systems  approach  to  training  and  instruction  (referred  to  as  Instructional  Systems 
Development,  ISD)  usually  consists  of  Analysis,  Design,  Development,  Implementation,  and 
Evaluation  phases.  The  ISD  approach  emphasizes  (1)  specific  behavioral  objectives  that 
define  exactly  what  the  student  is  expected  to  be  able  to  know  or  do  at  the  end  of  instruction, 
(2)  links  from  these  objectives  to  tasks  that  are  required  for  the  job,  and  (3)  ways  of 
verifying  that  the  student  has  in  fact  acquired  thit  skill  or  information.  The  process  typically 
starts  with  an  Analysis  phase  in  which  the  curriculum  designer  interviews  or  observes 
expert  job  holders  or  looks  at  the  design  of  equipment.  This  phase  produces  an  inventory  of 
job  tasks--that  is,  an  exhaustive  list  of  the  skills  and  knowledge  that  are  required  for  the 
job.  In  the  Design  phase,  which  follows,  the  tasks  to  be  trained  are  selected,  converted  into 
learning  objectives,  and  sequenced  into  a  particular  type  of  instruction. 

In  the  current  Navy  ISD  method,  there  are  two  paths  for  gathering  the  necessary 
information  for  the  Analysis  and  Design  phases. 

1.  PPP/TPS.  This  path  starts  with  the  production  of  personnel  performance  profile 
(PPP)  tables.  A  PPP  table  is  a  "minimum  listing  of  all  knowledge  and  skills  required  to 
operate  and  maintain  a  system,  subsystem  or  equipment,  or  to  perform  a  task  or  function" 
(DoD-HNDK-292,  p.  5-3-1).  These  tables  are  produced  by  means  of  generic,  task  analysis 
model  statements  provided  in  the  handbook,  with  the  appropriate  level  of  detail  for  that  job  or 
rating  added  by  the  curriculum  developer.  These  PPP  tables  then  feed  into  the  training  path 
system  (TPS),  which  is  a  system  for  selecting,  at  the  proper  level  of  training  (from 
apprentice  to  expert),  the  PPP  tasks  that  will  be  part  of  the  course  of  instruction  under 
development.  The  criteria  for  PPP  task  selection  is  guided  by  the  handbook,  which  also 
provides  steps  for  converting  the  tasks  into  behavioral  learning  objectives,  both  course-level 
learning  objectives  and  detailed  topic  learning  objectives,  and  then  sequencing  this  material 
into  a  course  outline. 

2.  MPTAR.  The  other  Analysis  and  Design  method  offered  in  DoD-HDBK-292  is  the 
manpower,  personnel,  and  training  analysis  report  (MPTAR),  and  this  method  requires  a 
similar  set  of  steps.  The  MPTAR  starts  out  with  the  job  task  analysis  summary,  which 
provides  a  complete  inventory  of  all  tasks  performed  on  the  job.  The  second  step  is  the  job 
training  task  list,  which  lists  the  subset  of  the  job  task  inventory  that  is  to  be  performed  by 
the  trainee.  Two  manpower  reports  (manpower  planning  data  and  manpower  summary) 
provide  the  number  and  type  of  personnel  required  to  do  the  job.  The  job  training  analysis 
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matrix  provides  a  matrix  breakdown,  by  subsystems  and  equipments,  of  the  skills  required. 
The  final  section  of  the  MFTAR  is  the  training  analysis  summary,  and  this  document  uses 
the  output  of  the  previous  ones  to  produce  course  and  topic  learning  objectives  in  a  course 
outline  structure. 

During  the  ISD  Development  phase  the  actual  training  materials,  for  both  the  trainee 
and  instructor,  are  produced.  According  to  the  handbook,  the  material  is  based  on  the  topic 
learning  objectives  but  with  more  detail  and  with  an  overview,  examples,  practice,  summary, 
and  tests.  The  handbook  specifies  the  development  of  instructor  and  trainee  guide  materials, 
and  testing  materials  for  the  assessment  of  students. 

The  final  ISD  phase  is  Implementation  and  Evaluation,  where  the  instructors  are 
trained,  and  the  materials  used  and  revised  as  necessary.  Although  the  handbook  specifies 
less  for  this  phase  than  for  the  previous  phases,  this  phase  is  often  one  of  the  most  time 
consuming,  since  courses  will  be  taught  for  many  years,  during  which  time  the  equipment  and 
its  operating/maintenance  procedures  will  be  upgraded  and  changed  numerous  times. 
Curriculum  materials  maintenance  and  revision  is  an  immense  problem  for  any  large 
organization  such  as  the  U.S.  Navy. 

As  others  have  noted,  the  ISD  approach  is  also  generally  followed  in  more  traditional 
instruction  (Montague,  1979).  What  the  systems  approach  adds  to  less  formalized 
instruction  is  descriptive  detail  and  the  possibility  of  better  quality.  This  systems  approach 
may  seem  straightforward  enough,  but,  for  courses  of  any  length  at  all,  the  management  and 
development  tasks  needed  to  document  and  complete  the  training  materials  are  quite 
imposing.  For  instance,  a  course  with  200  hours  of  relatively  uncomplicated  classroom 
instruction  (five  to  six  weeks),  with  perhaps  1000  objectives,  will  typically  take  from  10,000 
to  20,000  hours  for  development  and  at  least  18  months  for  completion. 

During  initial  revision  and  subsequent  maintenance  of  the  course  content,  it  is  often 
necessary  to  change  the  material  to  reflect  updated  equipment,  policies,  or  procedures,  and 
moreover  to  trace  those  changes  all  the  way  back  to  the  analysis  and  design  work.  Clearly, 
this  sort  of  audit  trail  is  crucial  in  identifying  any  parts  of  the  curriculum  development 
process  that  may  be  deficient  and  is  also  necessary  for  person-hour  accounting,  budgeting, 
and  scheduling.  However,  revision  and  course  maintenance  tracking  has  been  done  almost 
entirely  without  the  benefit  of  automation,  and  current  manual  methods  may  average  as  much 
as  $40,000  per  year  for  a  representative  10- week  course. 

Limited  Office  Automation 

The  managers  and  developers  in  Navy  training  activities  have  few  up-to-date  office 
automation  tools  to  do  their  jobs  more  easily  and  efficiently  .  This  almost  total  absence  of 
computer-based  tools  to  assist  in  the  complex  and  time-consuming  tasks  of  curriculum 
development  and  maintenance  slows  down  the  development  and  maintenance  of  training 
materials.  Moreover,  the  complete  design  and  logistics  process  is  becoming  more  computer- 
based  under  Computer-Aided  Acquisition  and  Logistics  Support  (CALS),  the  DoD-wide 
initiative  to  provide  for  the  electronic,  digital  delivery  of  engineering  information  for  new 
weapons  systems  from  contractors  to  government.  Thus,  the  need  for  automated  tools  for 
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curriculum  development  will  become  even  more  crucial.  Among  other  things,  the  lack  of  office 
automation  in  curriculum  development  has  resulted  in  training  materials  that  do  not  reflect 
the  technical  changes  in  fleet  equipment  and  in  instrucrional  graphics  that  are  difficult  to 
produce  and  change. 


Purpose 

The  overall  goal  of  AIM  is  to  provide  the  Navy  with  automated  tools  for  the  design, 
development,  production,  and  maintenance  of  curriculum  materials.  Computer-based  tools 
will  allow  scarce  subject  matter  and  instructional  design  experts  to  produce  high  quality 
instruction  efficiently.  This  report  describes  our  status  in  automating  the  mechanics  of 
document  preparation,  thus  increasing  the  efficiency  of  curriculum  writers.  We  also  oudine 
our  plans  for  enhancing  the  quality  of  training  materials  in  planned  software  tools  foi  AIM. 

Approach 

In  this  section  we  will  outline  our  approach,  identify  test  site  involvement,  and  describe 
the  developmental  hardware  configuration. 

Rapid  Prototyping  Methodolocv 

Our  approach  in  this  project  was  one  of  rapid  prototyping  -  that  is,  putting  computer 
systems  and  the  prototype  software  tools  into  the  hands  of  Navy  curriculum  developers  as 
soon  as  possible.  This  process  involved  an  iterative  design  and  development  process  in 
which  the  application  software  was  installed,  tried  out  by  the  users,  modified  on  the  basis  of 
problems  and  suggestions,  and  then  installed  and  tried  out  again.  This  interactive 
development  contrasts  with  the  more  traditional,  top-down,  structured  approach  where  all 
user  requirements  are  specified  in  the  first  phase  of  the  approach,  and  only  then  does  the 
software  programming  begin.  A  rapid  prototyping  approach  often  leads  to  software  being 
fielded  when  not  yet  in  standard  working  condition,  but  this  approach  allows  the  final 
functionality  needed  for  the  application  to  be  more  completely  specified  and  good  software 
engineering  principles  to  be  followed  in  the  development  of  the  final,  production-level 
software. 

Test  Sites 

From  the  very  beginning,  we  involved  actual  Navy  training  activities  with  day-to-day 
responsibilities  for  the  design,  development,  revision,  and  maintenance  of  training  course 
materials  in  the  project.  To  this  end,  several  test  sites  that  represent  the  different  types  of 
Navy  training  activities  were  selected  to  evaluate  AIM’s  effectiveness  (Fig.  1).  In  FY85, 
the  Training  Systems  Development  Department,  Naval  Education  and  Training  Support 
Onter,  Pacific  (NETSCPAC)  and  Engineering  Systems  School,  Service  School  Command, 
Naval  Training  Center  (NTC)  Great  Lakes  became  the  initial  test  sites.  In  FY87,  a  system 
was  installed  at  the  Fleet  Training  Center  (FTC)  San  Diego;  and  the  system  at  NTC  Great 
Lakes  was  expanded  to  other  parts  of  Service  School  Command. 
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Fig.  1.  AIM  Test  Site  Locations 


In  late  FY87,  an  AIM  test  site  was  established  at  Newport  News  Shipbuilding  (NNS), 
where  the  AIM  system  was  provided  to  a  NAVSEA  contractor  to  investigate  the  interface  of 
the  AIM  prototype  software  with  an  integrated  logistics  support  database  on  a  large  IBM 
mainframe  computer.  In  this  effon,  the  training  developers  worked  with  the  logistics 
engineers  to  extract  the  necessary  training  information  from  the  standard  logistics  support 
analysis  record  (LSAR)  data  bases  and  then  to  use  this  information  in  curriculum 
development.  This  effort  will  continue  through  FY89. 

In  early  FY88,  AIM  test  sites  were  also  established  at  the  Navy  Submarine  School 
(SUBSCOL)  New  London  and  the  Trident  Training  Facility  (TTF)  Kings  Bay,  GA.  Further 
test  site  expansion  also  occurred  as  a  result  of  discussions  with  OP-1 1  and  Strategic 
Systems  Programs  (SP-IS).  AIM  systems  were  installed  and  are  now  in  use  at  o^er 
Trident-related  activities:  TTF  Bangor,  and  SP-15  and  PMS-396  offices,  Washington,  DC. 
Test  sites  were  also  established  during  FY88  at  AEGIS  Training  Center,  Naval  Ship 
Weapons  Systems  Engineering  Station  (NSWSES)  Port  Hueneme,  and  Naval  Ship  Systems 
Engineering  Station  (NAVSSES)  Philadelphia. 
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The  installation  of  the  AIM  software  at  each  test  site  required  the  acquisition,  support, 
and  integration  of  the  necessary  off-the-shelf  hardware  and  software.'  These  acquisition 
decisions  were  made  to  satisfy  current  and  emerging  standards  in  DoD  and  Navy 
computing.  These  standards  included  UNDC,^  a  non- vendor-specific  operating  system; 
transmission  control  protocolAntemet  protocol  (TCP/IP),  a  set  of  DoD  communications 
standards;  and  portable  operating  system  interface  for  computing  environments  (POSK),  a 
federal  information  processing  standard  for  software  applications  portability.  Also  the  CALS 
intertranslatability  specifications  for  text  (Standard  Generalized  Markup  Language,  SGML) 
and  graphic  (Interim  Graphics  Exchange  Standards,  IGES)  were  considered  (MIL-HNDK- 
CALS  Draft,  1988).  The  hardware  (Fig.  2)  was  chosen  competitively  to  meet  these 
requirements. 


Fig.  2.  Typical  AIM  System  Configuration 


1.  Identincation  of  specific  hardware  and  software  systems  is  for  documentation  only  and  does  not 
imply  any  endorsement 


2.  Unix  is  a  trademark  of  AT&T  Bell  Laboratories. 
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The  AIM  prototype  software  was  designed  and  developed  in-house  by  the  AIM  project 
team  to  assist  Navy  curriculum  developers  in  the  preparation  of  training  documents  to 
support  the  Navy’s  standards  (MIL-STO  1379C  and  MIL-HDBK-292)  for  curriculum 
development.  The  hardware  consists  entirely  of  off-the-shelf  items:  Sun  Microsystems 
Unix  workstations/servers,  a  laser  printer,  ASCII  video  terminals,  Ethernet  and 
telecommunications  equipment,  graphics  and  text  (OCR)  scanners,  XT-  or  AT-level  PCs 
(such  as  the  Zenith  Z-248),  hardcopy  color  output  devices  (8-pen  color  plotters  and  35nim 
film  recorders),  and  other  associated  components. 

The  off-the-shelf  software  consisted  of  text  editors,  paint  programs,  and  computer- 
assisted  drafting  (CAD)  programs  (CADKey  and  TekniCAD)  for  color  graphics  production. 
Additionally,  some  systems  integration  work  was  required  at  each  test  site,  including 
integrating  these  computer  systems  through  networking,  with  both  local  area  netwoiks 
(LANs)  and  wider  ranging  connections,  such  as  DDN  (Defense  Data  Network)/MILNET. 

Status 

This  section  describes  the  software  system  architecture  as  it  evolved  from  prototype  to 
its  current  state.  Use  of  this  prototype  software  at  the  test  sites  for  the  production  of  course 
materials  is  given.  Also,  a  brief  overview  of  the  work  underway  on  expanding  the  software 
capabilities  is  provided. 

Prototype  AIM  Software  Capabilities 

The  AIM  software  was  designed  to  help  Navy  curriculum  developers  in  the  preparation 
of  training  materials  to  support  the  Navy’s  standards.  Contractors  deliver  curriculum 
materials  according  to  the  standards  for  military  training  programs  (MIL-STD  1 379C),  and 
the  Navy  training  activities  use  the  additional  guidance  given  in  the  military  handbook,  DoD- 
HNDK-292.  Each  AIM  software  module  supports  and  automates  much  of  the  document 
preparation  for  curriculum  development  (Fig.  3). 


Fig.  3.  AIM  Prototype  Software  Supporting  MILSTD  1379C 
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In  the  first  years  of  the  project,  we  completed  the  major  part  of  determining  the  actual 
operational  procedures  used  to  produce  Navy  instructional  materials.  These  data  were  used 
for  the  development  of  the  prototype  authoring  system  and  for  the  design  of  the  overall 
automated  system.  In  1^86,  the  primary  components  of  the  prototype  system  that  were 
completed,  including  software  for  development  of  PPP  tables  and  of  the  instructor  guide  and 
the  trainee  guide.  In  FY87,  the  prototype  software  system  was  expanded  to  provide  for  the 
capability  of  producing  some  of  the  TPS  documents  and  for  completely  formatted  trainee  and 
instructor  guide  documents.  Also,  basic  user  documentation  and  on-line  help  functions  were 
added.  In  FY88,  we  began  work  to  expand  and  revise  these  components  into  a  production- 
level  system  with  all  of  the  major  components  of  MIL-STD  1379C,  and  with  maintenance 
and  revision  capabilities.  The  completely  integrated  software  system  is  being  developed 
primarily  by  a  software  support  contract  that  was  awarded  in  FY88. 

A  brief  description  of  the  capabilities  of  the  AIM  prototype  software  follows.  (See  DoD- 
HNDK-292  for  a  more  complete  functional  description  of  the  Navy  curriculum  development 
process.) 

User  manual.  Allows  curriculum  developers  to  learn  the  AIM  programs  in  a  self-paced 
mode.  The  manual  guides  the  users  through  the  basics  of  the  UNIX  system  and  each  of  the 
curriculum  development  programs. 

PPP  program.  Assists  curriculum  developers  in  the  development  of  PPP  tables.  For 
system,  subsystem,  and  equipment  PPPs,  the  program  uses  the  model  statements  from 
DoD-HNDK-292  so  that  "boilerplate"  PPPs  are  automatically  generated;  for  background  and 
task/function  PPPs,  the  basic  table  structure  is  created  but  the  user  must  fill  in  the  details. 

Table  assignment  matrix  (TAM)  program.  Assists  curriculum  developers  in  the 
general  organization  of  the  course  to  be  developed  and  in  the  development  of  TAM  tables. 
After  asking  general  questions  about  the  level  of  training,  this  program  allows  the  users  to 
choose  PPP  tables  for  this  course. 

Training  level  assignment  (TLA)  program.  Assists  curriculum  developers  in  the 
development  of  TLA  tables.  The  program  allows  users  to  select  PPP  line  items  to  be 
included  in  the  course  development  and  assigns  a  training  level  based  on  the  user’s 
response  to  a  series  of  questions. 

TPS  printing  program.  Allows  curriculum  developers  to  edit  and  print  out  the  entire 
training  path  system,  including  the  introduction  and  training  objective  statements. 

Course  structure  program.  Allows  curriculum  developers  to  modify  the  automatically 
generated  course  structure,  which  is  then  used  in  the  programs  listed  below. 

Topical  outline  program.  Allows  the  curriculum  developers  to  create  and  edit  the 
topical  outline  for  a  course. 

Topic  learning  objectives  program.  Allows  curriculum  developers  to  create  the 
course  learning  objectives  and  to  modify  the  topic  learning  objectives  (TLOs)  generated  from 
the  TLA  program. 


-9- 


Instructor  guide  program.  Allows  curriculum  developers  to  enter  material  necessary 
for  the  Instructor  Guide,  including  the  front  matter,  back  matter  and  an  automatically 
generated  table  of  contents. 

Trainee  guide  program.  Allows  curriculum  developers  to  enter  material  necessary  for 
the  trainee  guide,  including  the  front  matter  and  an  automatically  generated  table  of  contents. 

Master  materials  list  (MML)  program.  Allows  curriculum  developers  to  create  and 
edit  the  master  materials  list. 

Graphics  generation  and  text-graphic  integration.  The  CAD  programs  (TekniCAD 
and  CADkey)  have  IGES  (Interim  Graphics  Exchange  Standards)  translators,  which  allow 
translation  of  the  graphics  into  and  out  of  neutral  formats.  Thus,  a  drawing  created  on  one 
software  system  can  be  viewed  and  modified  in  the  other.  These  CAD  graphics  files  can  also 
be  translated,  through  in-house  developed  graphic  filters,^  into  Postscript  or  Sun  raster  files 
for  black-and-white  output  or  for  page-by-page  integration  with  the  text  material  being 
generated  on  the  Sun  systems.  There  are  also  utilities  to  translate  graphics  files  from  TIFF 
(tag  image  file  format)  to  Sun  raster  and  to  allow  the  inclusion  of  Sun  raster  files  with  the 
textual  material.  Another  capability  under  development  allows  the  production  of  35mm 
slides  from  the  CADkey  or  Sun  raster  format. 

Results 

Some  of  the  production  using  the  AIM  software  is  listed  in  Table  1.  To  date,  the  AIM 
software  has  been  employed  to  help  produce  approximately  64  courses,  totaling  about  500 
weeks  of  instruction.  AIM  test  sites  generally  report  time  savings  of  from  20  to  50  percent, 
with  better  and  more  consistent  instructional  materials  resulting  from  this  system.  Some 
representative  courses  that  have  been  produced  are  listed  below,  and  the  overall  use  of  the 
prototype  software  is  given  in  Table  1. 

•  Electrician’s  Mate  "A"  School 

•  Tactics  and  Maneuvering  Fundamentals 

•  Heat  Stress  "F"  School 

•  Mess  Management  Specialist  "C"  School 

•  Weapons  Technician  "C"  School 

•  Propulsion  Engineering  Basic  Course 

•  Gas  Turbine  Mechanical  and  Electrical  Courses 

•  FFG-7  Class  Waste  Heat  System  Maintenance 

•  Cargo/Weapons  Elevator  Maintenance 


3.  Copies  of  these  software  documents  may  be  requested  from  the  authors. 
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Test  Site 

Number  of  Courses 

Weeks  of 
Instruction 

NTC  Great  Lakes . 

. 20 . 

. 398 

NETSCPAC . 

. 21 . 

. 76 

FTC  San  Diego . 

. 15 . 

. 19 

SUBSCOL . 

. 3 . 

. 4 

TTF  Kings  Bay . 

. 1 . 

. 2 

TTF  Bangor . 

. 4 . 

. 2 

Table  1.  Test  Site  Production 


While  working  with  users  at  each  of  the  test  sites,  we  learned  much  about  the  user 
interface  and  functional  requirements  of  this  software  during  its  iterative  development.  This 
process  thus  helped  us  determine  the  format  for  screen  displays,  strategies  for  on-line  help, 
style  of  user  manuals,  and  the  underlying  database  structures  necessary  for  reliable, 
reusable,  maintainable  software.  All  of  this  information  was  then  used  in  the  design 
specifications  for  the  new  complete,  integrated  system  (see  last  section). 

Discussion:  Prospects  for  AIM  and  for  Better  Training 

We  staned  this  report  by  describing  the  current  problems  that  the  Navy  faces  in 
curriculum  development  for  its  classroom  courses.  Now  we  will  examine  how  the 
implementation  of  an  integrated  AIM  system  can  help  solve  these  Navy  problems:  a  large 
inventory  of  courses  developed  by  scarce  personnel  and  taught  by  instructors  not  familiar 
with  the  demands  of  technical  training;  a  complex  curriculum  development  and  maintenance 
process  that  is  guided  by  changing  production  standards  and  that  provides  few  electronic 
tools.  These  characteristics  about  the  Navy’s  approach  to  training  materials  development 
will  likely  remain  part  of  the  way  that  the  Navy  does  business,  even  20  years  from  now.  But 
many  of  these  problems  could  be  eased  by  the  use  of  automated  tools  for  curriculum 
development  such  as  those  that  will  be  included  in  the  overall  AIM  system. 

1.  Dispersed  courses  connected  electronically.  The  Navy  teaches  over  7200 
courses  among  its  widely  scattered  shore-based  schools.  By  networking  various  training 
activities  over  dial-up  phone  lines  or  the  Defense  Data  Network,  people  who  are  developing 
curriculum  materials  at  widely  scattered  training  activities  will  be  able  to  trade  information 
and  to  access  remote  and  local  databases  of  technical  information  and  course  materials. 

Thus,  instructional  materials  will  become  more  standardized  and  the  process  will  become  far 
more  efficient.  Establishing  this  sort  of  electronic  user  community  for  various  specialties  and 
interest  groups  within  the  Navy  should  greatly  reduce  the  duplication  of  effort  and  probably 
will  save  millions  of  dollars. 
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2.  More  effective  use  of  scarce  personnel.  The  Navy  does  not  have  enough 
technical  and  educational  experts  to  adequately  design,  develop,  maintain,  and  teach  the 
classroom  training  materials.  AIM  will  provide  tools  that  allow  for  the  more  efficient  capture 
of  subject  matter  expert  knowledge  and  skills,  and  thus  these  scarce  technical  experts  will 

be  better  used.  By  giving  technical  and  educational  experts  better  tools  for  curriculum 
development,  they  will  have  much  better  control  and  influence  over  the  delivery  of 
instructional  materials  to  the  students. 

3.  Automated  tools  reduce  expertise  needed  for  curriculum  development.  Military 
curriculum  developers  are  not  usually  given  the  training  or  the  time  to  acquire  the  educational 
expertise  necessary  for  developing  effective  technical  instruction.  The  current  prototype 
AIM  software  helps  to  automate  the  formatting  and  production  requirements,  and  later,  more 
capable  versions  will  provide  embedded  training  on  the  procedural  guidelines  of  DoD- 
HNDK-292  and  guidelines  for  good  educational  practice.  Thus,  personnel  who  have  limited 
experience  with  instructional  design  will  be  able  to  design  and  develop  more  effective  and 
efficient  training  materials. 

4.  Software  designed  for  changing  standards.  In  the  past  40  years,  the  Navy’s 
procedural  guidelines  for  curriculum  development  have  changed  many  times,  and  the  software 
architecture  underlying  AIM  is  designed  to  be  able  to  cope  with  surface-level  changes.  By 
separating  the  data  from  the  format  and  by  creating  databases  that  underlie  the  curricula  and 
its  technical  content,  AIM  will  allow  developers  to  access  and  share  resources  even  as 
standards  change.  This  separation  of  data  and  format  will  be  increasingly  important  as 
contractors  start  to  deliver,  along  with  new  weapons  systems,  text/graphics/design 
databases  in  digital  format.  Under  CALS,  these  text  and  graphics  files  must  be  packaged  in  a 
neutral,  device-independent  format. 

5.  Software  tracks  complexity  of  instructional  development.  The  curriculum 
development  process  is  a  complex  and  time-consuming  one;  the  Navy  guidelines  (MIL-STD 
1379C  and  DoD-HDBK-292)  are  lengthy  and  complicated  documents;  the  handbook  is  in 
two  volumes  totalling  over  500  pages.  The  AIM  curriculum  development  tools  can  help 
manage  this  process  by  tracking  the  decisions  made  during  curriculum  development  and  their 
subsequent  effects.  For  instance,  automation  aids  to  task  analysis  and  learning 
requirements  specification  can  improve  quality  and  increase  transfer  to  job  performance. 

Also,  by  using  local  area  networks  as  well  as  wide  area  networks,  and  by  providing 
specialized  software  programs  for  the  management  and  development  of  training  materials, 
AIM  can  make  the  complex  work  environment  much  more  manageable. 

6.  Better  office  automation  environment.  Navy  curriculum  developers  have  few 
office  automation  tools  to  accomplish  the  tasks  of  instructional  development.  The  AIM 
software  tools  will  create  a  "netwoiked  publishing  environment"  and  with  graphics  and  text 
tools,  the  actual  developers  will  be  able  to  produce  more  effective  and  efficient  curriculum 
materials.  The  AIM  software  includes  such  programs  as  drafting  programs  for  technical 
illustration,  a  set  of  lower-level  writing  tools  (spell  checkers,  synonym  databases,  and 
sentence  level  prose  processors)  integrated  into  a  specialized  document  preparation  package 
for  Navy  training  materials  development. 
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Next  Steps 


The  Navy  faces  a  number  of  challenges  in  meeting  the  requirements  of  producing  and 
managing  its  classroom  training  materials.  To  meet  these  demands,  the  prototype  AIM 
software  is  undergoing  substantial  revision  into  a  production-level  system.  This  more 
complete  system  will  support  all  of  the  Navy’s  core  curriculum  development  components, 
such  as  both  analysis/design  paths  (PPP/TPS  and  MPTAR),  student  achievement  testing, 
and  automatic  generation  of  tracking  documents.  Moreover,  because  of  its  underlying 
relational  database,  the  production-level  AIM  will  facilitate  maintenance  of  curriculum 
materials  over  their  long  life  time.  Thus,  the  eventual  AIM  system  will  allow  Navy 
curriculum  developers  to  efficiently  produce  standardized  training  materials  that  are 
technically  accurate  and  consistent,  to  maintain  this  technical  accuracy  by  updating  the 
electronic  databases  underlying  the  course  materials,  and  to  distribute  these  text  and  graphic 
training  materials  over  local  and  wide-area  networks. 
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